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This three-^part volume provides a summary, for use by 
practitioners, of a project concerned with h<Jw novices learn to 
become creative educational computer users. The' first chcHprtTer, 
examines techniques for increasing the novice's understanding " o.f 
computers and computer programming, and specifically analyzes the 
potential usefulness of five recommendations for the design of 
computer literacy curricula. For each recommendation , ^a statement of 
the issue, an example, relevant background, and a brief review of the 
relevant 4 research literature are provided. In the second chapter, a 
framework is presented- for describing users' knowledge of how a 
simple '4-f unctidn calculator operates, and data on individual . , 
differences among novices and experts in their conceptions of a 
calculator's internal operations for varioijsj sequences are 
summarized. The ttiird chapter provides a summary of a study in which 
30 undergraduates learned BASIC programming language through a 
self-paced mastery manual and we're tested on their Mmantal models for 
the execution of ea' of hine BASIC statements. A catalog is then 
presented of beginning programmer's conceptions of the internal 
functions of the computer for each* of tjhe BASIC statements based on 
test Results. Each section inciudes tables and references. (LMM) 
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Note 

The material in thfs chapter has. been published as the following article: 

Mayer, R. E. Contributions of cognitive science and irelated research £n 

learning to the design 'of coraputejpHf^teracy curricula* >In R. J. 

Seidei; R. E. Anderson, & B. Hunted (Eds^K Computer Literacy: 

Issues and Directions for 1985 . New York: Academic Press, 1982, 
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Mayer, R. E, Contributions of cognitive science and related research on 

learning to the design of computer literacy curricula, . Technical 
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The goal of. this paper is to examine techniques for increasing the 
y novice 1 ! understanding of .computers and computer ^programming. In particular, 

ttife paper examines the potential usefulness of five recommendations "concern- 
ing the design of computer literacy curricula, as listed below. 
. . : i (1) Provide, the learner with a coacrete model of the computer. 

^ /* (2') Encourage the learner to actively restate the new technical 

; i * j 

. s 

information in his or her 'own words. * * 

> . (3) Assess the leaVner's extsting v intui tionS about computer operation 
jand try to build on them, or modify them, as needed. x 
^ [k) Provide the learner with methods foj* chunking statembnts Into a , 

larger, single,, meaningful unit. 

(5) Provide the learner wi th methods for analyzing statements into 
smal ler, meaningful parts. 

V 

For each recommendation, this paper wi 1J provide a clear statement pf the ( 
issue, an example, relevant background, and a brief review of relevant 
research literature. 

« As can be seen, each recommendation^ i s concerned with increasing the 

meaningfulness of learning new computer information by^ novices. For pur- 

• o 

poses of the present paper, meaningful ^earning ,is viewed as a process In 
which the learner connects new material with knowledge that already exists 
in memory (Bransford, ,1979) . Figure 1 provides ,a general framework for 
discussing the conditions of meaningful learning (see Mayer, 1975, 1979a). 
^The figure shows ^that information enters the human cognitive system from 
the outside (e.g., through text or lectures, etc.), and must go through the, 
following steps: CO Reception . Fj'^t, the learner must pay attention to- 
the incoming information so that it reaches working memory, as indicated 
by arrow a. (2) Availability / Second, the 'learner must possess appropriate 

ERIC . ' , 6 
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prerequisite concepts In long term memory to use in assimi lating the new 

information, as indicated by point b, '-(3) " Activation , Finally, the 

♦ * * * 

learner must actively use this prerequisite knowledge during learning so' % 
tftat the new material may be connected^ with it, as indicated by arrow c ; 
from long term memory to working memory. Thus, in the course of meaningful 
learning, the learner must come into contact v/ith the new material (by 
bringing it into working memory)., then must search long term memory for 
what Ausubel 0968), calls "appropriate anchoring ideas 11 , and then must 
transfer those ideas to working memory so they can be combined with the new 
information in working memory. . Each recommendation is aimed aft insuring 
one or more tf these .conditions is met. • ■ 

The traditiona-1 way^pf eval ua ting meaningful learning is ^to test 
whether learners can transfer what they Have .learned to new situations. For 
example, Wethelmer (1959) taught students how to find the area of a parallel- 
ogram using* a rote method (j.e., memorising a formula) or a meaningful method 
(i.e., involving the structure of the figure): Although both groups performed 
equally well on problems like those'given during instruction, Werth^n^^ f 
claimed Jthat the meaningful learners were able to solve unusual problems 



requiring creative transfer. Thus, this paper will focus on transfer as. a 
measure of meaningful learning of computer programming. 



1. » 'USE CONCRETE MODELS 

Statement of the Problem m 

^ 'Novices tend to lack domain specific knowledge (Greeno, 1980; Simon, 
1980; Spflidr, Vesonder, Chiesi 6 Voss, 1979). Thus, one technique for 
improving the novice's understanding of new technical information is to'" 
provide them with a domain-specific framework that can be used for assimi* 
lating new information-»~Le. by allowing for "^ailabi 1 i ty" as indicated 



by pdint b In Figure 1. The present section focuses on the effects of con- 
crete models . on people^? understanding of; computers .and computer programming. 
Example 

For example,- in our own work on teaching a simple BASlC~like language 

* f ~ 

to novices, we presented a model of the computer such as shown in Figure 

2. Thejnodel provides concrete 'analogies for four functional units of the 

computer: 0) input is represented as a ticket window in which data is 

linda up waiting to be processed and is placed in the finished pile after 

being processed, (2) output is represented*as a message no,te>adwith one g 

message written per line,* (3) memory is represented as an erasable scoreboard 

in which there is natural destructive read-in but non-destructive read-out, 

and 00. executive control is represented as a recipe or shopping list with 

c 

a pointer arrow to indicate the line being processed. This model may be 
presented to the learner either as a diagram, or as an a6tual board contain- 

ing these useable parts. ^ . /* 

Background * 

' There Is ample evidence that concrete models are widely used in mathe- 
matics instruction. For example, early work by Brownell. Moser 09^) 
indicated "that children who' learned subtraction, algorithms with the aid of 
M bundles -pf sticks" were better able to. transfer to new problems than 
children who were given the rules for subtraction. in. abstract form with 
plenty of "hands on 11 experience in executing the procedures. More 
recently; the Important role of "mani pulatives" such as coins, sticks, 
blocks, etc., has been documented by Weaver & Suydanj (19.72) and by Resnick 
& Ford (1980).. \ 

There is also some evidence that concrete models may enhance compre- 
hension of text.. For example, students' recall of an ambiguous passage 
was enhanced when a title or diagram- or introductory sentence was given 
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prior to reading but not when given after reading (Bransford S Johnson, 

i 

1*972; Dooling & Lachman, 1971; Dooling S Mullet, 1973)* Similarly, Ausubel 

(1963, I960, 1968} has provided some evidence! that expository learning may 

be enhanced by using mT^advance organizer 11 — a short, expository introduc- 
ed 

t ion ^presented prior to the text, containing no t speciflc content from the 

^ ' ' *** ' \ " " 

text and providing a^general framework for. subsuming the information in 

the text. More recent reviews of the'advance organizer literature reveal 

thafadvance organtzersr tend to have their strongest effects in situations 

where learners'are unlikely to already possess useful prerequisite concepts- 

namely, for technical or unfamiliar material, for low ability subjects, 

and when 'the test involves transfer to new situations (Mayer, 1979a, 1979b).. 

^ ' ' ! 

Royer and his colleagues (Royer & Cahlej 1975, 1976) have demonstrated 

that concrete models may serve as effective advance organizers in learnln<g 

new^cientif ic information. For example, the concrete analogy of electrical 

conduction as a chain of falling dominoes, Influenced subseq&STTfcyl earning. 

Similarly, White & Mayer 09.80), analyzed the concrete model's used by 

physics textbooks. For example, Ohm's Law is described i 7 n terms of water 

flowing, in pipes, a boy pushing a heavy load up an inclined street, cr 

* * 

electron flow in a circuit. Recent results by Cook & tiayer 0sQ6)' show 
that. when concrete analogies are embedded in a technical text, novices tend 
"t)o perform be # st on recalling these familiar models and tend to recognize 
the information related to the models. 

DuBoulay and his' col leagues (DuBoulay & O'^hea, 1976; DuBoulay, O'Shea 
& Monk, 1980) have distinguished between two approaches 'to learning computer 
programing, tn the black box approach, the. operations of the computer are 
-hidden to the learner so. that the learner has no idea of what goes on inside 
the computer. In the glass box approach^, the user is able to understand 



the changes that occur inside the computer fof'each statement. Although the 
description need not, indeed should not,- be on a machine language lev^l, * 
DuBoulay et. al. ( 1980) suggest two properties for making the hidden oper- 
ations of a computer language more clear to the novice: sfmpl ici ty— there 
should. be a "small number-of parts that Interact in ways that can be 
easily understood 11 , and visibility— novices should be able to "view „ 
selected parts and processes of this notational machine in action 11 . 
DuBoulay et* al. have implemented these suggestions in an instructional 
course in LOGO, since each statement is related to a concrete model called 
"the LOGO machine 11 . -However, there is yet no empirical test concerning the 
effects of the LOGO machine on learning. * 
Research of Concrete Models 

Transfer. In order to provide some Information concerning the role of 

* v 

models on learning computer programming, a series of studies was conducted 
(Mayer, 1975). In the studies, subject's were either given a concrete model 
of the compute^, (such as shown in Figure ,1) or not;' then, all subjects read 
a 10-fD.age manual describing seven BASIC-l'iRe statements (see Table l). 
Following reading, subjects took a test that consisted of si* types of 
problems (*»s shown in Table 2).. For generate problems, the subject had to 
write a program; for interpret problems, the subject had to describe what 
the program would "do. . 1 

*The proportion correct response by type of problem is given in the top 
of Table 3. As- can be seen, the control cjropu performs 'wel 1 , on problems 
•that are very much'l'ike the material in the instructional text, e.g\, 

. generate-statement and generate-noaloop. However, on problems that require 
moderate amounts of transfer— e.g. , generate-loop and the shorter interpret 

•"•problems; the model group .excels. This difference in the pattern of 



performance suggests that models enhance transfer performance but not 

>^ ■* °» o * 

Simple retention of presenced material. Apparently, the*model provided an 
assimilative context in which novices could relate new technical infonr.a- 
*. tion'in the'booklet to a familiar anaVogy. This assimilative process 
resulted in a broader learning outcome that supported moderate* transfer. 

Locus of the effect. One 'problem with the above study is that the - 

' 1 * 4 

model subjects received more information than controls. Therefore, 

# / 

another series of studies was conducted (Mayer, 1976a) in which all sub- 
jects read the same BASIC-like manual, but some* subjects were given a 
concrete model of the computer before reading while others were giysn the 
same* model aft'er reading the manual. 

The proportion correct response |>y type of problem for the two^ 
groups is shown in the bottom of Table 3. 'As in the previous study, the 
before^ group excels ^on creative transfer to new situations but the after 
gYoup excels on simple retention of the .presented material. Thus, as 
predicted Jby assimilation theory, the model serves as an assimilative 
cqntext for learning only if it is available to the learner at the time 
of learning. m " € . 

Recall. The above studies used transfer tests as measures of what 
is learned under different instructional treatments. In a follow-up study 
(Mayer & Brgmage, 1§80), subjects read the manual and were given model 
either before or after the reading as in the previous study. However, as 

a test, subjects were asked to recall all they could about certain portions 

* ». 

* of the manual . # 

in order' to score the protocols', the Information in the manual was 
broken down into idea -.units. Each idea unit expressed one major idea or 
action. Thereiwere three kinds of idea .units in the manua.l : (11 conceptual 
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idea units related to the i/fiternal operation of the computer, (2^* technical 

« * 

idea* Units gave example? of code, and (3) format idea units gaye grammer 
rules. Table k gives examples of^each type of idea unit. 

.Table 5 shows the^verage number of idea units recalled from each • 
category by the two groups. , As can be seen, the before group recalls more 

conceptual^ information while the after *group excels on recall of technical 

* * * «« 

and format information. Thi$! pattern is consistent with the ideagoctf 
retention requires recall of specific code, but good transfer ^requj res 
understanding of conceptual tdeas. Also, the 'before group included more v 
intrusions about the model and other sections, of the manual, sugtfes.ting 
they integrated the information more broadly. * / 

Different language. Although the 'above results are consistent and 

were obtained in a long senes of studies, their generality Is limited 4 . 

* " \ - ' \ 

by the fact that just one type of language was? Used. Thus, a follow-up 

study was condupfced .'(Mayer, lS$0a) in which subjects learned a file manage- 
ment language (Gould & Ascher, 197 2 *). either with or without a c6ncrete 
modek Table 6 lists the eight statements' that were described in tir , 
instructional manual. Fi gure 3-shows the concrete model that was used: 
long-term memory is ..represented as ^ffle cabinet; the sorting function, is „ 
represented as an in-basket, out»bisket and save basket; temporary memo r, 
Is "rep resented as an erasable scoreboard; executive control is representee! 
as a list and pointer arrow; output is represented as a message pad. -After 
instruction, all subjects .took a transfer test including simple retention-like 
problems {sont-1)and problems, that required putting all of |he learned^,-- 
commands together in a novel way, (compute- J and 2). (See Table 7.) - 

Table 8 g'ives the proportion correct response by type of problem for 
the two treatment groups. As in the previous studies, the control group . 



"performs best^bn simple problems like 'those in the manual, but the model 
grodp excels on longer problems^at require creative integration. Thus > 
previous- iresul ts and conclusion seen) to generalize to thi$ new domain. 

Ab i 1 i ty . .^he pattern of result described above tended to be strongest 
for low ability subjects (Mayer, 12175) whece ability is defined fn terms 
of Mathematics SAT scores. Apparently, high ability learners already 
possessed their own useful "models 11 for thinking about how a computer works, 
but low ability students would be more likely to lack useful prerequisite 
knowledge. ' * 

s Text organization . The pattern of results described above also 
tended to be strongest when material was poorly organized (Meyer, 1978). 
Apparently, the model is more useful when material is poorly structured 

* 

because it helps the reader^to hold the information together. 

Evaluation . These results provide-clear and consistent evidence that 
a concrete analogical model can have a strong effebt^gn the encoding of 
new technical information in novices. These results provide empirical 
support to the claims of DuBoulay & O'Shea (J 976, 1978) that allowing 



novices to Msee the works 11 allows them to encode information in a more 
coherent and useful way. When appropriate models ar*e used, the. learner 
seems to be able to assimilate each new statement to his or her image of 
the computer system. Thus, one straightforward implication is; if your 
goal is to produce learners who will not need to use the language creative- 
ly, then no model is needed; if your goal is to produce learners who will 
Be able to come up with creative solutions to novel (for them) problems, 

then a concrete jnodel early in leaVning is quite useful. More research Is 

/ 

needed in order to determine the specific effects of concrete models on 
whaj! is learned, and to determine the characteristics of a useful mbdel. i 
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2. ENCOURAGE LEARNERS TO "PUT iT IN THEIR OWN WORDS" 
Statement, of the Problem 

A second technique for increasing the meahingfulness of technical in- 
^formation is elaboration— encouraging the learner to explainthe informa- 
tion in his or her own words and to verbally relate the material to other 

« 

concepts or ideas. Elaboration techniques may influence meaningful 
learning: because they encourage, the activation of existing knowledge that 
is relevant for comprehending the presented material — i .e. elaboration % 
may affect theVctivation process as indicated the arro\£^lfn Figure 1. 
Examp 1 e - . 

For example, in our" own research, we have taught Subjects a simple file 
management language as described in the previous section Csee Tables 6 and 
7). in order to encourage subjects to elaborate o^^re materia 1, we pre- 
sented questions after each page of the instructional booklet. Table 9 
gives examples of M mbdel elaboration questions 11 which ask the learner to 
relate the material to a familiar context, and ''comparative elaboration 
questions 11 which ask the learner to relate one part of the material to 
another. , , 
Background 1 

There is some evidence that asking subjects to put ideas into their 

own words \during learning can enhance the breadth of learning. For example 

i * , 
Gagne & Smith (1962) found that subjects who were required to give a 

\ 

verbal rationalization for each move as they learned to solve a n.ew problem 
resulted in longer learning time but, better transfer performance than non- 
verfaalizers. Results by Seidel & Hunter (J970). suggest that verbalization 
per se may not significantly enhance computer programming performance. 

More recently, Wittrock (1974) has proposed the .."generative hypothesis 
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.i.e.. learning occurs when the .learner actively gene/a tes associations ( 
• between what Is presented and what he or she already has in memory. For 

. example, when school children wex.e__asked £6 generate a one-sentence "sum- 

4 " ~ / I 
mary for each paragraph in a prose passage, Wi ttrocfcs.f 197*0 found that ^ 

recaM was nearly double that of a control group. Apparently,, when stu- 
dents are encouraged to actively put inprmation In. their own words, tbey 
are able to better cpnnect new information to exfsting knowledge. \ 

Elaboration techniques have long been used to enhance learning of 
pafred associates. For example, when students are asked to actively form 
images or sentences involving word pairs, paired associate recall ts 
greatly- enhanced (Bower, 1972; Pa.ivib, 1969). More recently, elaboration 
techniques have been used in school curricula (see -Dansereau, 1978; • 
Weins'tein, 1978},. Several researchers have argued that students should be 
given explicit training in- learning strategies 0 --! .e. how to actively 
process new material Csee O'NeMJ, 19781. ■ < 

Transfer . In a typical study, (Mayer, 19.80a) subjects read .the instruc 
tional booklet covering a simple file management language, with some sub- 
jects having, an elaboration page after each page in the Booklet (model 
elaboration), and others not (control),. A second study followed the same 
procedure but there w$s a comparative elaboration page aft^r each page 
for half the subjects. % * 

On a .subsequent transfer test, using problems described tn Table 7, 
the control groups pe/formed well on simpler retention-like problems but 
the elaboration groups (both model and comparative), perform better on pro- 
blems requiring creative' transfer. Table 10 shows the proportion correct 
by type of problem for four treatment groups. Thus, tfrere fs evidence 
that requiring the learners to put technical information *in their own 

', . ,15 
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words through, relating the material to a familiar situation or through mak- 

* I 

ing comparisons, results in broader learning. 

Reca 1 1 . In order to assess the -genera 1 i ty^o'f these findings, the 
studies. were replica.t.ed using recall as the test (Mayer, 1980a) . For ^ ^ 

t 

'scoring, the manual was divided into idea units. Some idea units described^ 
•how the computer operated (conceptual idea units) and others emphasized the 
grammar and technical aspects of each statement (technical idea units). ,' 
Table 11 shows the average number of idea units recced by type for 
model elaboration,, compar ison^elaboration,. and control grpups. As can be 
seen, the control group tends to recall equal amounts of both types of info'r- 

'rnatioq, but the elaboration groups tend to emphasize reca 1,1 of conceptual 

T *'■ • ' 

'as compared to technical information. This pattern is consistent with. 

o 

the idea that conceptual emphasis is likely to support transfer performance-, 
o . . 

Notetaking . In order to -pr ovide* f urther ; ge neral i ty, an additional series' 

of studies was conducted (Peper & Mayer, 1 $78). using a different language 

(a BASIC-like language) and, a different elaboration act-m (note-taking) . 

Subjects watched a 20 minute videotape lecture describing seven BASIC-like 

statements similar to the manual "described earlier. Some subjects were asked 

to take, notes by putting thebasic information in their own words. Others 

' o 1 

srmply viewed the lecture without' taking notes. As a test some subjects were 

given transfer problems and some were asked to recall portions of the lesson. 

As in preyious studies, there was a pattern in which note-taking improved 
performance on far transfer problems but not on pimple retention problems. 
Similarly, there was a pattern in which note-takers performed better on recall 
of conceptual information but not technical information. These patters were 
observed for. subjects scoring low in Mathematics SAT, but not for high ability 
subjects. Presumably high ability learners already possess strategies for 

16 • , • 
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putting new mTormation into their own words* 

Evaluation * Unfortunately, there is no fopl-proof way to design elabor- 
ation activities. However, it is important to keep in mind that the goal of 
elaboration is to^help the learner be able to describe the key concepts In 
his or her own words, using existing knowledge. Emphasis on format or 
grammatical details, and emphasis on e» rorless ^verbatim recall of statements 
wi;ll not produce the desired effects. The learner should be able to 
descri'be the effects of each statement in his or her own words. 

3. ASSESS AND BUtLD ON LEARNERS 1 ' INTUttl.ONS 
Statement of the Problem 



Learners come to the learning situation with certain .existing exp'efcta- 

tions and intuitions about how to Interact with computers* Far exaqjple, 

since studjents have experience with conversations in English,* they are 

likely to try to view computer conversations in the same way (Hi Her & Thomas, 

» 

1977; Sackman, 1970).- Similarly,' since most users are ' fami 1 iar- wi th aslcu- 
!lators, they may view interactions with computers' i-n the same way (Mayer & 
. Bayman, 1980; Young, 1980). Q 

Ex ampl e - , 

■ v 

For most Users, calculators represent the first exposure to interacting 

O * 
with a computational machine. Thus, Intuitions that- are established may be 

important for later learning of computer programming languages'. For example, 

consider the keystrokes: * *~ ' * 

If subjects have a conception of incrementing internal registers, they 
might suppose that this sequence would result in^U being displayed, however* 
less sophisticated Intui tions. might predict that the display would show 7 
or 0. • 1 7 



Background # ' f 

* 

-There Is a growing interest in using wor^s. and logical structures. that 
are similar to everyday English* Tor exampl e, Ledgard, Whiteside, Singer 
S Seymour (1980) found that text editing systems that use . n natural language", 
are easier to learn than those that use "computerese" for commands. 
Similarly, Shneiderman (1980) reports that meaningful or mnemonic variable 
names may affect programming performance. Finally, there is evidence that 
branching structures used in BASIC are not as intuitive or as easy to 
learn as other branching structures (Green, 1977; Mayer," 1976&; Sime, Green 
S Guest, 1977; Sime, Arblaster & Green, 1 9.771 . 

More recently, Young 0980) has developed "mental models" of calculators- 
i.e. representations of the internal components -hat a learner needs to 
understand, Scandura, Lowerre & Veneski (1976) have interviewed children who 
learned to use caltulators. through "hands on experience". Many develop , 
bizzarre intuitions even though they can use the calculator to solve routine 
problems. Tl)us, in order to build on the learners intuitions, and modify 
them as needed*, one must assess what those intuitions are, in other words, 
the instructor should have techniques for determining the learner's "mental 
model". , ' "I"'-. 

Analysis of Users 1 Intuitions of Calculator Operations . 

A series of studies was conducted* (Mayer & Bayman, 1980) in order to 
determine the intuitions that novice and' expert users have cbnfierning how* 
pocket calculators ^operate. The novices were college students with no 
experience with computers or computer programming, while the experts were 
intermediate level computer science students. Each subject was given a 4-page 
questionnaire with 88 problems. Each problem listed a series of key presses 
and asked* the student to predict what number would be in the display, 
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assuming a standard £our-f unction calculator .was being used. 

The subjects differed greawy with respect to when they thought "an 
expression should be evaluated. For example, consider the*problems, 

2+3 

2 + 3 + .'._.« 

2+3+7 • / 

2 + 3 + 7^, 

/ ( Some subjects behaved as if an expression was evaluated only when an equals 

' was pressed; thus, the answers were 3> 3, 7, 12. Others behaved as if an 

♦ • 

expression 'was evaluated as soon as^an operator key was pressed, yielding 
answers of 3, 5, 7, 12. Finally, some subjects behaved as if an expression 
. * was evaluated as soon as a number was- pressed, giving answers of 5,^5, 12, 12. 
Insults indicated sKgnificant differences between experts. and novices, wfth 
most experts opting for second approach while novices were *f airly split 

* r 

among all three approaches. There also were imponlant differences concerning 

h6w to evaluate a chain of arithmetic, such as, 2 + 3x7=, and how to^handle 

non-standard sequences^uch as 2 + - + =. 

Evaluation . We are just beginning 'to develop techniques for describing . 

» 

users' intuitions, "e.g., users' mental models of computational machines. 

However, as techniques become. avai lable, teachers jnay' use them to diagnose 

* * * w^, * * * 

whether students have acquired useful intuftions, and to remediate where needed. 

h. PROVIDE TRAINING IN CHUNKING 

t 

Statement of the Problem 

■ ■ * « 

One technique for making storage of information easier is to form meaning- 

chunks of schemas (Bransford, 1979) • Within the context of computer 

programming, this means that learners should develop the ability to view a 

% \ 
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cluster of statements as a single unit that accoirfpl fshes some namable goal. 

* * — _ ' 

v Example 

, % For example, Atwood S Ramsey (19-78) suggest that experienced program- 
mers encode a segment such as, 
SUM - 0 

'■> * . * 

DO 1 1 = 1, N 

SUM = SUM + 01 

1 CONTINUE ' j 

as "CALCULATE THE ,SUM-,0F ARRAY X". 

Background 

TheVe is some evidence, that experts and novices, in a particular domain 
differ with resp'ect to how they organize information .in memory, with experts 
♦ using more efficient chunking techniques (Larkin, McDermot't, Simon & Simon, 
lg8o). In recent reviews" of research on how to teach people to become 
better problem solvers, Greeno (1980). and Simon O980}, conclude that good 
problem .solving performance requires that the user" has large amounts of 
domain specific knowledge organized into chunks. For example, Simon (I98O) 
estimates that a person needs 50,000 chunks of domain specific knowledge 
(e.g., such as the example gtven ab.ove) to become an expert. 

In a classic study, Chase 6 Simon 0973) asked subjects to view 
briefly presented chess board configurations and then try to reconstruct 
them. Chess masters performed better than less experienced players in re- 
constructing positions from actual games, but the advantage was lost when 
random board positions were presented. Inan analogous study reported by 
Shneiderman (1980), experienced and inexperienced programmers were given £ 
programs to study. The experts remembered more than the novices when actual 
programs were presented but not for random Vines of code. These findings 

o -21) ■ ' ■ 
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suggest that experts have a large repertoire of many meaningful chunks, i.e. 

. \ r . , 

ways of grouping many lines of code into a, single meaningful unit. More 
recently, Mayer (1979c, 1980b) has suggested that highly used 'chunks, such 
as lopping structures, should be explicitly taught and labeled as part of 
instruction. For ^example, frequent looping structures in BASIC include 
"repeating a READ", "wai ting, for *a* data "number", Waiting for a Counter 11 , 

r 

and "b'ranching down". ^ 



5. PROVIDE TR'AfNING T'N ANALYSIS OF STATEMENTS 
Statement of the Problem , 

« 

What does i t mean to "understand 1 ' a statement 11 In many psychol inguisticf 
theories, comprehension involves relating a statement to its underlying case 
granmar (see Kintsch, .197^L 
Example v 

In a previous paper (Mayer, 1979c}., I have suggested a possible case 
grammar for BASIC. Each statement may be described as a list pf transactions. 
A transaction consists of an action applied to some object at Some location 
ffij the coniputer. For example, the statement, LET X = 5, consists of s4x 
transactions: 

1. Find the number indicated oh the right of the equals. 

2. Find the number" in the memory .space ijndicated on the left of the equal 

3. Erase the number fn that memory space. „- . 
k. Write the new number in that Space. 

f ♦ 

5. Go on^to^the next statement. ^ 

6. Do what i t says; 
Background 

. * An ImpHcation of the "transaction" approach is that the same statement' 
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names may actually refer to several different types of transactions. For 
example, we have shown that a counter set LET 'such as LET X = 5 is differ- 
ent from an arithmetic LET such 'as LET X - 10/2 (Mayer, 1979c/ 1980c). 
Explicit naming and describing of different types of statements with the 
same keyword may become a useful part of computer literacy curricula. More* 
•recently this, approach has been successfully applied t^the analysis of 
commands in ''calculator language 11 (Mayer & Bayman, 1980) and text editor" 
languages (Card, Moran & Newel 1 , M98O) T ' . \ 

• •• • ' 1 • 

CONCLUSION • ' 

T&is paper has provided five tentative recommendations, listed in 
the i t ntroductio>u for increasing the meaninpfqlness of computer concepts 
for novice's. ^Reviews of cognitive research indicate that there is .qualified 
support for the first two recommendations, and that active research is needed 
concerning the latter three recommendations. J 
Note 

l.wish to thank Bob Seidel and Ron Anderson for their useful comments on an 
earlier version of this manuscript. A more detailed version of this paper 
is available as a technical report from the author.' Much of the work cited 
in this paper was supported by grant SED77-19875 from t;he National Science 
Foundation and grant N!E~G80~01l8 from the JaJtional Institute of Education. 
Requests for reprints should be sent to: Richard E. Mayer, Department of 
Psychology, University of California, Santa Barbara, CA" 93106. • 
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-Table 1 , 
Seven Statements. Used hi BAS'lC-like Instructional Booklet 



Name 
READ 
WRITE" 
EQUALS 



CALCULATE 

GOTO 

IF 

STOP 



Examp 1 e 
PI READ(Al) 
P2 WRITE (Al) 
P3 Al =-88' 



?k Al = Al + 12- 

# • 

P6 go to pi 

P5 .IF (Al = 100) GO TO P9 

P9 STOP 
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Table 2* . • * 0 * 

Examples of Six Types of Test Problems for a BASlC-like Languag 

Generation-Statement * 1 n terpreta t i on-S ta tement 

Given a number in memory space A5 = 0 
A5, write a statement to change 
that number to zero, : 



Genera tion-Non loop # >! 

Given a card with a number 
on it is input, write a 
program to print out its 
square. 



In terpreta tion-Non loop 
pr .READ (Al) 
?Z Al = Al * Al 
P3 WRITS (Al) 
?h STOP 



Gen ej a t i on - Loop ? ng 

Given ^a^jp i 1 e of data cards 

is input, write a program to 



print out 'each number and stop 
when it gets to card with 88 
on it. * 



Interpretation-Looping 

PI READ (Al) 

P2 IF(A1 - 88) GO TO P5 

-El^WRTTE (Al) 

?h GO TO PI ^ ^ 

P5 STOP / 
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. Tgble 3 ' ' * 

* « 

"Proportion Correct on Transfer Test by Type of Problem for Model vs. Control Groups, and Before vs After Groups 

* • * 

" Generation 



, Interpretation 



Statement Nonloop 



Model vs* Control 

Model \63 
Control * .67 



• 37 \ 
.52 



.Looping 



.1o 



.12 



Statement 



.62 

M 



T 



Nonloi 



op Loop i ng 

. j 



..62 
.32 



.09 
.12 



* Before vs. After 

•Before .57 
After .77 



.50 
.63 



.20 
.13 



•i»7 

.27 



.63 



..17 
.17 



Note.' For model vs. control, n ■ 20 per' group; Interaction between treatment and problemi^T^e, p < .05. 
For before vs. after, n = 20 per group; interaction between treatment and$p|^lem type, p < .05. 
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Examples of Test Problems for a" Eilc Management Language 



Sort 1 



List the owners ) names for all 
cars weighing 3000 pounds or more. 



FROM AUTOMOBILE ✓ 

FOR WEIGHT IS CALLED 3000 OR MORE 

LIST NAME 



Sort 2 



List the owners' names for all late 



model green Fords. 



FROM . AUTOMOBILE . 
FOR YEAR IS CALLED 1976 OR MORE 
AND FOR COLOR IS CALLED GREEN 
AND FOR MAKE IS CALLED, FORD 
LIST NAME 



Count 

o 

How many cars are registered in 

Santa Barbara County? 



FROM AUTOMOBILE 

FOR HOME COUNTY IS CALLED SANTA BARBARA 

COUNT 

LIST COUNT 



Compute 1 •> 

What is 'the average current value 

t 

of all cards^ 



FROM AUTOMOBILE * , 

COUNT *. 

TOTAL CURRENT VALUE 

LET TOTAL * COUNT BE CALLED AVERAGE 

LIST AVERAGE . 



Compute 2 

What percentage of 1977 cars are 
. Chevrolets? 



FROM AUTOMOBILE . . - 

FOR BEAR .IS CALLED 1977 

COUNT 

LET THIS BE CALLED COUNT 1 

AND FOR MAKE IS CALLED CHEVROLET 

COUNT , 

LET THIS BE CALLED CQUNT 2 

LET' COUNT. 2 * COUNT 1 BE CALLED AVERAGE 

LIST AVERAGE . 
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Table 5- . 

Average Number of t .Recalled Ids^ Units' for the Before and After Croups 



Idea Units 



Intrusions 



Before 
After 



Technical Format Conceptual 

6.6 



5-0 
6.0 



2.*9 



Inappropriate Appropr 1 a te Model' 
1.5. 1.3 3.1 



'k.S. 



2.5 



.8 



.5 



Note. N « 30. per group; Interaction between treatment and type of score, p < .05. 
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* - Table 6 

Eight Statements Used in File Management Language Booklet 



3 



Name 
FROM 
FOR 

AND FOR 

OR FOR 

LIST 

COUNT 

TOTAL 

LET 



Example 

FROM AUTOMOBILE 



FOR HEIGHT ZS CALLED 3000 OR MORE 




AND FOR COLOR IS CALLED GREEN 



OR FOR MAKE IS CALLED FORD 
LIST NAME 

COUNT » 
• TOTAL CURRENT VALUE 
LET TOTAL f COUNT BE CALLED AVERAGE 
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Example of Conceptual, Format , and Technical Idea Units 
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Type 

Technical 

Format 

Format 

Conceptual 

Conceptual 

Technical 

Technical 

Conceptual 

Conceptual 

Conceptual 

Conceptual 

Conceptual 

Conceptual 



I dea Unit 

f 

READ is one kind of statement. 

The format is READ ( )\ 

An- address name goes in the parenthesis. 

An address name is a space in the computer* s memory. 

Thefe are 8 memory spaces. 

The spaces are called Al, A2 •••• 

An example is, READ (A2) . - 

First, the computer checks the number from the top data card 
» Then, that number is stored in space A2. 
The previous number in A2 is destroyed. ( ^ 

Th£n the Sgta card is sent out of the computer. 
This reduces the pile of data card by 1. 
Then, go on to the next statements. 
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Table 8 



Proportion j^rect on Transfer Test for Model and Control Groups- 

File Management Language 



Mode] 
Control 



Type of Test Problem^ 
Sort-l Sort-2 Count jComputer-1 Compute-2 



.66 . .66 
,63" " .hk 



.63 



.58 
.33 



.45 
.22 



Note. N - 20 per group; treatment x problem type interaction, p < 



Table* 9 

Example of the Elaboration Exercise in the Programming Text 
M odel Elaboration 

Consider the following situation. An office clerk has, an Jn-basket, a save 

basket, a discard basket, and a sorting arfea on the desk. The in-basket is 

*• 

f uj 1 of records.. Each one can be examined individually in the sorting area 
of the desk and then placed in either the save 6r discard basket. Describe 
the FOR statement in terms of what operations the clerk would perform using 
the in-basket, discard basket, save basket, and sorting area. 

Comparative Elaboration 

How is the FOR command like the FROM command? 

How is the FOR command different than the FROM command? 
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Table 10 t 
Proportion Correct on Transfer Test by Type of Problem for Model Elaboration vs 
Control Groups and Comparative Elaboration vs. Control Groups 

Type of Test Problem 

Model vs. Control , Sort-1 Sbrt-2 * Count Compute- 1 Computer-2 
Model Elaboration ,.65 .55 .6** .64 .45 
Control- .66 .6* M .38 .27 

. Comparative Elaboration 
Comparative Elaboration .90 .90 1.00 .75 .55 
Control • ..90 .90 .65 .65 .25 

Note, For model* elaboration vs* control, o -« 20 per group; treatment 
x. problem t^pe interaction, p < .05. For comparative 

elaboration vs. control, i = 13 per group; treatment x 

< 

problem typtf interaction, p < .05* Data is for inter- 
pretation problems only, for comparative and control groups. 




Table 11 . * 

. Average 'Number of Recalled Idea Units for Model Elaboration, 
Comparative Elaboration and Contrpl Groups 

Type of , Idea Units' - 
Technical Conceptual 
Model Elaboration . 3 # 3 ' 13.9 

Comparative Elaboration S.k ]k.] 

Control 7.5 , 7,5 

Note. N = 20 per group; treatment x type . interaction, p < .05 for low 

♦ «■* . 

abl 1 i ty subjects. 



INPUT 
FROM . 
PERCEPTION 





(a) 

». 




SHORT-TERM 
MEMORY ' 
(STM) 




WORKING 
MEMORY 
(WM). ' 


M 





'(0 



LONG-TERM. MEMORY 
(LTM) 

(INCLUDES MEANINGFUL 
CONCEPTS-(b)) 



* " v ~ T A f . . .» 

Figure 1: Some information processing components of meaningful learning. Condition 
(a) is transfer off new knowledge to WM. Condition (b) is availability of £ 
assimilative content in LTM. Condition (c) is "activation and transfer of 
— _ _oM_knowledge to WMl ' 
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MEMORY SCOREBOARD 



A1 

7 


A2 

0 


A3 

99 


A4 

6- 


A5 

33 


A6 

2 


A7 

0 


A8 

3 



INPUT WINDOW- 
-V- 




PROGRAM LIST & 
POINTER ARROW 




OUTPUT 
PADa 



Figure 2: A concrete model of the computer for a BASIC-like language 



FILE CABINET 



, AUTOS, 



STUDENTS 




BOOKS 



IN BASKET 



SORTING AREA 



SAVE 



DISCARD 



OUTPUT PAD 



PROGRAM LIST 



POINTER 
* ARROW 
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MEMORY SCOREBOARD 



♦COUNT! 


• TOTAL1 


AVERAGE! 


COUNT 2 


TOTAL2 


AVERAGE2 


COUNT3 


TOTAL3 


AVERAGE? 


COUNT 


TOTAL 


AVERAGE • 



Figure 3. A concrete model of tfie computer fox a file management language;* 
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■ CHAPTER 2 

DIAGNOSIS AND REMEDIATION <{l BUGS IN CALCULATOR LANGUAGE 



\ . 



Note . 

The material in, this chapter was published as the following article; 
Mayer, R.. E. , & Bayman, P# A psychology of calculator languages.: 

Describing computer concepts to novices. Communications of the 
Association for Computing Machinery , 1981, 2ft> 511-520. • \ 



Abstract 

9 m 
ihis paper presents a framework for describing users 1 knowledge of how a 

simple four-function calculator operates* Data are summarized concerning 

differences * among novices and %xpert;s in their conceptions of "what goes on 

inside the calculator' 1 for various sequences of button presses. Individual 

0 

differences include: different view on when an expression is evaluated, 
different procedures for evaluating a chain of arithmetic, and different 

rules for evaluating unusual sequences of key presses. 

, - > 

s 

Key Words and Phrases: calculator, learning, instruction, psychology 
CR Categories: 1*50, 2.12, 4.20 



A major goal of this article is to provide a framework for describing 
users' knowledge of calculator language, i.e., users 1 intuitions concerning the 

r 

underlying logic of a sii^le four-function calculator when a series of buttons 
are pressed. Another major goal is to use this technique for pinpointing some, 
of the differences among actual users in their knowledge of calculator language. 
The first section of this paper provides a rationale and a brief literature 
tevieW. The second section describes the transaction approach for analyzing 
calculator language. . The third section summarizes a study of individual dif- 
ferences among users in their knowledge of calculator language. The final 
section provides a summary and a set of tentative recommendations. . ^ 
Rationale * ' '* < 

Performance versus competence . A basic idea in this* article is that the 
traditional distinction between performance and competence can and should be 
'applied to users/ learning of calculator language. Performance, of course, 
refers to what the user c&n dc>, such as compute answers for a class of problems; 
competence refers to what the user knows , such as the user's mental model of 
the calfiulator. . 

It is possible for two users to both give identical answers to simple , 
arithmetic computation problems, but possess vastly different underlying knowl- 
edge of calculator language. For example, "in pilot studies we found that .a 

\ 

subject believed the calculator had the answer for each problem already stored 
in memory. This subject would claim t&at to finS the answer for 22 x 12,4, the 
calculator simply "looks up" the answer for that problem in its memory. Another 
subject assumed that the calculator used f, internal registers, 11 and followed 
certain "control procedures. 11 /For 22 x 114, the calculator would store, the 

numbers 22 *fnd 114 in memory, and would use the multiplication algorithm to worl£ 

• t * * 

A. 

on them/ These two subjects seem to have had different "mental models" for the 
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calculator. Thus, for a complete description of "what is learned 11 .by different 
users, we must be able to describe the users 1 competence as well as their per- 
formance. Similarly, Greeno [3] has argued for emphasis on "cognitive objectivisY 
of instruction rather than focusing solely on "behavioral objectives" of instruction. 
Black box versus glass box . A second important distinction conderns^ 

4 " * 

learning by memorizing versus learning by understanding [15]. When we .apply 
this distinction to the learning of calculator language, we can point to the • 
difference between the "black box approach" and the "jglass box approach." In 

the black box ap^roaich to learning calculator language, £he user focuses only on 

* * 

the external features of calculator language — you put in a sequence of key 
presses and out comes the answer as if by magic. The operations in'side the 
.calculator are hidden from the user, forcing the user to treat th£ calculator as 
a black box that cannot be understood. A user who learns by the black box 
method is forced to memorize sequences of key presses for each type of probl^ta, 
without understanding what the key presses actually mean. For example, some 

manuals describe how to use a constant. * Let's say you want to multiply a set of 

o 

numbers by 2.3. The manual may instruct you to enter the sequence 2.3 x x; 
then, for any number you want multiplied by 2.3 you just: enter that number 
followed by atf eqyals (=). Although memorized procedures, such as the "con- 
stant" sequence, may work in the sense that they generate the* desired answer, 
the user is not able to relate the sequence of key strokes to an, understanding 
of what g&es on inside the calculator. DuBoulay S O f Shea [1] have noted a 
similar phenomenon -with respect to children learning LOGO; some users act, a6 if 
the internal operations of the machine are hidden and not understandable. 

In the glass box approach [2],, the user is able to see how a sequence of 
key strokes is related to change in the internal state of the calculator, and 
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how these changes are related to the fiii&l answer* Each command — in this case, 
each key stroke — results in some change inside the calculator and these changes 
can be described and understood. For example, the user who learns by the glass 
box approach may be able to describe why the "constant 11 procedure tforks, by 
describing the nature of internal displays and incrementing operations. 

The level of description of events in the glass *box approach need not,\ 
indeed shoul^ not, be at the "blood and guts 11 level* By this we'm^an that users 
need ijot become electronic experts'. There is an appropriate, level of descrip- 
tion that Young [16] refers w t& as the user's "mental model" of the calculator: 
"For an interactive system to be satisfactory, *it is important that its intended 
users be able to form a modei of the system ..which enables them to predict its 
behavior." • 

DuBoulay, O'Shea & Monk [2] have suggested that noVices should be exposed to 
a "notational machine" — i.e. "an idealized model of the computer implied by the 
constraints of the programming languages" and which is analogous to "other raech- 
anisms with which the novice, is more, familiar"., As an example, duBoulay O'Shea 
[1] have developed a "LOGO machine," to represent the internal actions that occur 
for, LOGO statements. Further, duBoulay, 0 T Shea & Monk [2] have offered two * 
important properties for selecting a model that clarifies the hidden operations , 
of -a language.: (1) simplicity — there should be a, "small number of parts that 
interact in ways that can be easily understood", and (2), visibility~~novices 
should be able to see "selected parts and processes of tfris notational machine in 
action". # 

As an example, let's suppose that we want students to, learn how to solve 
simple aJsithmetic rroblems v*i*-h a calculator. We could give thei^plenty of 
hands-on experience, without any guidance about what goes on inside the cal- 
culator, until they were all abl^-to solve simple problems. However, Scandura, 

4b 



ILowere, Veneski & Scandura [13] found that students who taught themselves to use 
calculators o£ten developed bizzare intuitions; one student, for example, con- 
eluded that the plus (+) and equals (-) keys did nothing since they caused no^ 
visible change in the display. Instruction that emphasises the understanding of 
how the machine operates on a sequence of button presses might provide a better 
base on which to^ build further computer concepts. 

What are the benefits of instruction that foster glass box learning* rather 

- j * . '/ 

tnan black box learning? Past research by Gestalt psychologists [15] suggest 
that learning by understanding, such as in the glass box procedure, should lead 

" O * 0 * 

to superior long-term retention and superior transfer to novel problems. In 
addition, the glass box approach,, may influence" attitudes concerning the "under- 
standability" of computers and calculators. Although** there is promising support 
for these assertions in studies of how novices, learn simple programmingylan- 
guages [4, 5, 6, 7, 9], much more research is needed concerning the role of 
glass box instruction for calculators. 

Computer literacy . The previous sections have presented two ways of 
describing "what is learned" (i.e., performance vs. competence) and two ways of 
teaching how to use calculators e., black box vs. glass box). Why is it 
important to focus on hg^fi^^dents learn ,and represent knowledge about calcu- 
lators? The reason is tha^/ calculators (as well as electronic games) usually 
involve a user's first exposure to a^gomputational machine and a language. 
Thus, calculators .could provide the first step in the development of a user's 
computer literacy — the understanding of how to interact with computational 
machines . * 

In addition, calculators have become a part of society, infiltrating the 
home, work, and school lives of ordinary people (10). In our schools, for 
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example* teachers [11] have recognized calculators as necessary tools in our 
. society: "The National Council of .Teachers of Mathematics recommends that 
mathematics programs take full advantage of the power of calculators and com- 
puters at all grade levels." However, in spite of the potential for using 
calculators' as the first step towards computer literacy, there is also the 



potential that they will be used as tools whose operations must be blindly 
memorized. For example, duBoulay, O'Shea &,Monk (2) -recently pointed out: 
"The manuals accompanying certain makes of pocket calculators make no attempt 
to explain the reason why given sequences of button presses carry out the 
given computations. The user must follow the manual's instructions blindly 
because it is difficult for him t6 imaging what kind of underlying machine 

, \ 

could be inside that demands these ^particular sequences .of presses. During 
the course of a calculation, he has, to guess the current state of the device.... 
because the device gives little or no external indications o£ its Internal" 
- state.-" • * * 

Unfortunately, the research community has been very slow in providing 
information that would be useful in this impending calculator-curriculum revo- 
lution. For example, most experimental studies have been concerned with whether 
using calculators in the classroom affects overall achievement and/or .attitude 
in mathematics [see 12, 14] j but as Roberts [12] recently concluded, "the re- 

search literature offers no guidance" cinceijpiing how to incorporate calculators 

* " \ 

into school curricula. * 

The development of a theory of how users conceptualize calculator language 

has implications for the design of calculator languages, for instructional 

procedures, and for integration of calculators into school curricula^ This 

paper is based on the idea that calculators are here to stay, that large numbers 

♦ • 
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of ordinary* (non-programmers) people will be using them, and that calculators 
provide most users with their first introduction to computer concepts. Such 
users will inevitably develop attitudes and approaches to human/computer inter- 
action in the course of learning to use their calculator even if the users are 
self-taught. This paper provides some information that may be relevant to 

understanding what intuitions individual users have about calculators. .Thus, 

♦ 

this paper is a start towards the goal of helping users to make the most of 

v their calculators "and computers in 'general. 

A Transaction, Analysis of Calculator Language 
* » » » 

The goal of this section, is, to develop an appropriate level of ^describing # 

what happens insidp the calculator for each type of key press, based on DuBoulay,. 

O f Shea & Monk's, [2], criteria of simplicity and visibility. In particular, 

1 ♦ 

this section, applies the transaction approach to the operating system of 
electronic calculators, or to what can be called calculator language. The 
gOaL is not to prQvide a formal, mathematical representation of the calcu- . \^7~. 
' lator f s operating system, but rather to provide an idealized model of the 
calculator that can be used to describe users 1 knowledge and to, help novices 

■o 

understand calculator language. „It should b<? pointed out that the trans- 
action approach may serve both 15 as (1) a descriptive model of the users 1 
knowledge of calculator language, and (2) a prescriptive model* for curriculum 
development. This paper presents data concerning the first implementation, but 
also suggests implications concerning the second. 

For purposes of this analysis ue assume that each user's conception of 
calculator language can be specified as a set of productions* or condition- 
action pairs. The condition refers to some key press (i.e., some command) and 
the action refers to one or more transactions (i.e., an operation applied to an 



V 

object at a location in the calculator). Thus, the transaction approach in- 
volves locating the transaction (or list of transactions) that a user 

c * i S 

associates with a given command, • . ^ 

A previous paper [6 J summarized a conceptual analysis of BASIC that em- 
phasized "transactions" for describing the language iit a simple and visible 
way, A model was constructed that consisted of a ticket, window to represent the 
input function, a note pad to represent' the output function, a memory scoreboard 
to represent the memory function, a scratch pad to represent the logic -and 
arithmetic function, a shopping list with pointer arrow to -represent executive 

* o * 

control. Each elementary BASIC statement was described as a list tff trans- 
actions , and each transaction consisted of some operation applied to some object 
at some location in the computer. Using a small collection of transactions it 
was possible to describe each of the elementary BASIC statements. Further, ^ 
there is substantial evidence that instruction in BASIC which emphasizes the 
transaction level of description, results in superidr performance in creative 
program writing and interpreting written programs [4, *5, 7, 9-]. 

The relevant conditions (or commands) for the "present analysis are based on, 
pressing keys on the calculator^ keyboard. The keys, relevant to a very simple 

4 

four-function calculator are number keys (i.e., 0, 1, 2, 3, 4, 5, 6, 7, 8, 9), 
operation keys, (i.e., +, x, *), equals key (-), .decimal key (.) and clear 
key (CLR) • For the present study we assumed that the calculator used aritlv 
metic lo^ic (rather than algpfaraic or» reverse Polish notation) and we focused, 
on only three number keys (i.e., 2,3, and 7), £wo operation keys (i.e., + 

a. 

and x), and -the equals key (=). * 
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!3)hus, at first blush it seems the basic conditions are each of the single key 
presses, such as pressing a number key, pressing an operation key, and so on. 
However, interyiews with users suggest that key presses have different "mean- 
ings ft depending on the immediately preceding key press; for example, pressing a 
plus key after pressing^a^umber key has a different effect than pressing a 
plus key after pressing an equals ^key, for some users. Thus, the conditions 
(or user commands) can be listed as some key being pressed given that some key 
was pressed immediately before. For example, typical commands in the present 
analysis are listed in Table 1. There are certainly many other, possible 
commuids, but we have focused on this set of 16 elementary calculator commands 
as an example. ~ / % 

■ y . — r— — - ■ 

Insert Table 1 about here 




To describe the actidlis that occur for any pommand, the transaction 

approach [6] requires that we specify the triplet- of location, object, and 

- / * * 

operation. The possible locations within the calculator are: 

I 

(1) Display — The external display normally consists of at leas' 
> eight spaces, where a place can hold one digit. The 

display fills from^the right/ 

(2) Register — An internal register is inside the calculator and con- 

sists of a series of subregisters Jihat hold individual, 
numbers and operators. • Expressions* a)ce held in the order * 
of input, with the first number of the left, followed * 
b^-first operator, ^nd witl\ new numbers and operators 
entered to the right of existed filled subregisters. 



(3) Keyboard — The external set of keys includes number, operation, 

** " ■ * *» 
and equals keys; " * \ 

tHe*pbssible objects include: ^ t , 

(1) Numbers — . A number is any single or Multiple digit sequence such, 

as Z, 14, or 156. ^ » 

• (2) Operation — An operation, is a mathematical symbol for some arittr- 

* 

?*etic computation such as addition (+) or raultipli- 

cation (if). , > 

• * * 

(3) x "Expression — ? An expression is a sequence .consisting of numbers and 

\ • * , * . • 

\ operators such as, 2 , + 3 or 2 + or 2 + 3'zz 7. 

\ . 

Some operation&^that are relevant to computer language are: \ 

- \ * • ■ 

(1) Find Locate a particular object; e.g., find a number that was 

just entered from the keyboard. 

(2\ Destroy — A number or expression is* erased from the display or 

regis^r; e.g., when you press the equals key the * 

* \ 
previous number in t^e cisplay is erased (and replaced 

„ with a new one) * - 

(3) Create — A number or expression is placed in a display or 

** » 
1 register; e.g., when you press a number key that 

* i 

number appears in the display. \- * 

(4) m Evaluation — An expression from the register is converted into a 

single number using the rules of arithmetic; 'e. g". , the 
v evaluation cf 3 + 2 %& 5.*' (For the current discussion, J 
evaluation of a number or numbers followed by an oper- , . 
ation is the number; i.e. evaluation of 3 is 3 or 
evaluation of 3 + is 3). It is also ? ^r*ssible to 
evaluate expressions from the register and display 
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* ■ together;, for example* the evaluation of 2 + in 

« 

the register and 3 in the display may be 5. 

v 

Table 2 gives a summary of some typical actions that might occur with, the 
calculator. Each is expressed as a list of transactions; for example, D = R 
consists of four separate transactions, while D = D requires only one* A user's 
conception of what^ a particular command means can be expressed as a production; 
for example, the production, 

P2 If # after + Then D = # and R = "R + #" 
means .that when number key'is pressed after a plus key, the user assumes that 
the calculator executes the four transactions for D - // and the. tjiree transactions 

« c 

for R = !'R + p" listed in Table 2. Thus, for the sequence 7 + 3,.when the 3 



Insert Table 2 about here 



keX-As pressed the display is changed to 3 and the regime? s expression is 
changed' to "7 + 3". A user t ! s intuitions concerning calculation language can 
thus l?e expressed as a list of productipns such as the one given above. 
Empiri-cal Studies 



There has not been adequate research concerQing>4low students come to 



4 

understand the operation of calculators. As N noted earlier^—almost all be- 




havioral research concerning calculators has been directed at the gross issue 
of whether^ the availability of calculators in the class/oom has any effect on 
mathematics achievement or attitude (see 11, 14] ^ The present study addresses 

a different issue, namely what types of hypotheses do people have concerning 

t * 

how calculators operate. The goal of this research is to determine whether 

* »• 
the transaction approach can be successfully used as a f ramework for de- 
scribing differences in users 1 knowledge. The goal of these studies is not 



to test the transaction approach -as a "theory," since it is used here only as a 
framework for describing what is learned. For more detail concerning the 
methodology and data analyses see Mayer & Bayman [8]# 

Met^tod . Our study involved 33 college students who had no computer pro- 
gramming experience ("novices") ^nd 33 college students who were enrolled in 

2 \* ' 

advanced programming courses ("experts") . Subjects participated in order 

to fullfill a crourse requirement* Each student was given a four page question- 
naire with 88 problems. Each problem listed a series of key presses and asked - 

the student to predict what number would be in the display, assuming a standard 

» * ■ 

four-function calculator was being used. In addition, subjections were given 

,a questionnaire asking hort many hours a week they used a calculator,, how many 

years they had been using a calculator, what kind of calculators they know. 

and which calculator model (s) they owned, if any* 
• * 

Standard sequences: Whefe to evaluate * Our subjects differed greatly with 
respect to when they thought an expression should be evaluated. For example, 
consider the sequence, 

i 

•2 + 3 

Some subjects answered "3" and some answered "5". Those who gave 5 seem to be 

using what we called "immediate evaluation" for if after +. Whenever a number 

" * * t 

key is pressed after a plus key, the entire expression is evaluated and dis- 
played. However, those who gave 3 as an answer seem to be using "delayed 
evaluation" for*# after +. They wait for some other key press (such as an 

\ * ; 

equals or a plus or a multiply) before they evaluate and display. How would a 
subject predict the calculator would respond to 
2 + 3+7 

The answer was 12 for the "immediate evaluators" and 7 for the subjects who 
relied on delayed evaluation for // after +. Our subjects were very consistent, 



although experts were significantly more 'consistent than novices^ in judgments 



such as these. 

Now consider the 



problem, 



2 + 3 + / * 4 

j 

. / 

Some subjects gave 5 as the answer while others gave 3. Those who gave 3 act 

as if there is "delayed evaluation" for + after 3. For those who gave 5 as 'an 

answer, if they also gave 5 as an answer to problems like 2+3, they are not 

evaluating for + after // either; however, if they gaVe 3 as an answer for 2 + 

3, then they seem to opt for "immediate evaluation" for + after it. Similarly, 

» * * 

a sequence like, ', 

2x3+ . 
results in 6 for subjects who rely on immediate evaluation for + after #, but 
in 3 for those who rely on "delayed evaluation" for + after //. (Note*that if 
our subject relies on immediate evaluation fot // after x then the answer will* 
also be 6.) - 4 
Finally, ^consider the problem, * 
2 + 3 = 

All subjects gave 5 as an answer* Or consider the problem, 

^ 2> 3 + 7 - • ' * ' 

All subjects gave 12 as an answer. However, if our subjects were delayed 
evaluation for // after +.and delayed evaluation for + after // then we know 
they wait for an equals sign before v ;they evaluate, thus, these subj&dfcs_yould 
opt for "immediate evaluation" for =? after //. 

Based on a systematic analysis of our subjects 1 performance, as suggested 
in the, examples above* we noted three basic strategies for determining when to 
evaluate an expression — for a number key, or for an operation key, or for an 
equals key* These 'conceptions are summarized in Table 3. As can be seen, the, 
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consensus of the experts is that a calculator should evaluate when an operation 
key is pressed after a number, as is common in most but not all calculators. 

Novices tend to have much more diverse conceptions, and are significantly 

3 • ' 

different from experts. It may also be pointed out that we found no relation 

* " 

between the conceptions *of our subjects and the operating systems of their own, 

calculators, nor between the conceptions of our subjects and the amount of time 

- * 4 # " 

spdnt each week with a calculator. » 



Insert Table 3 about here 



\ 



Standard sequences: Chains of arithmetic . How would you predict a cal- 
culator would answer., 

2 + 3x7 = 
How about the problem, 

2x3 + 7=- ■ 

l 

Our subjects varied with respect to how they evaluated a chain of arithmetic. 
The vast majority of subjects executed the operations in order from left to 
right, yielding answers of 35 and 13 respectively for the above' problems. 
Some subjects tended to opt for multiplication being carried out before addi- 
tion, yielding answers of "23 and 13 respectively. Some subjects tended to opt 
for addition being carried out before multiplication, yielding, 35 and 20 

» o 

respectively. Some subjects opted for carrying out the second operation first, 
yielding 23 and 20 respectively. Finally, some subjects simply ignored all but 
,the last computation, yielding 21 and 10 respectively. Table 4 summarizes the 
major strategies for evaluating a chain, based on an analysis of each subjects 
performance on several problems. As can be seen, most subjects opted for left- 
to-right evaluation of a cliain, although a substantial minority of experts 



^assumed multiplications were carried out before addition* This procedure is 
characteristic of some sophisticated calculators and computer commands* 



r 
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Non-standard sequences: Equals after operator * The foregoing two sec- 
tipns demonstrated that there are considerable differences among subjects 1 
interpretations of calciflator operations even for standard sequences of key 
strokes. A standard sequence is defined as one that begins with a number and 
in which an operator (like + or x) or equals (=) may only follow a number. In 

the present section we explore subjects' conceptions of how the calculator 

■* 

responds to non-standard sequences of key strokes* A non-standard sequence 

♦ *■ • 

violates the above "grammatical rule of arithmetic" by having two or more 
operators (+ or x) and/or equal sign (-) in sequence. Users' predictions 
concerning non-standard sequences are useful because they allow us to 'diagnose 
users ^conceptions of the internal operation of the calculator. 
For example, consider the sequence, ^ 

. . 7 + - * * ; 

or, consider the sequence, 

7 x . - 

How do subjects interpret the calculator's. operations? Some subjects assume 
that a non-standard sequence results in the display being reset; for example, 
if resetting the display means setting it to zero then subjects give 0 as the 
answer to the above problems. Another version of the reset strategy is to 
assume that the calculator will show an E in the display, or that it will flash 
on and off. A second group of subjects act as if the calculator simply ignores 

' V 

/ 

the ,non-standard sequence; in this case, the calculator display has 7 in it for 
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pach of the above sequences. Finally, a third major group acts as if a number 
has been inserted between the operator hnd the display; e.g., they treat 7 + = 
a*§ 7 + 7 = and give an answer of 14, or tliey treat 7x = as7x7 = and give an 
answer of 49. We call* these subjects "incrementing display" subjects because 
they act as if the number in the display is added to the number in the internal 
register. There are several variations on the incrementing display strategy; 
for example", 2 + 3 + = can result in 10 or in 8 depending on the subject's 
conception of when evaluation occurs. * 

« e * 

Table 5 summarizes these three major conceptions of what happens when equals 

follows an operation; as can be seen, the strategy of ignoring. the non-standard 
• ■ \ 

sequence is the most common but experts are far more likely to opt for the 
incrementing display conceptualization. The incrementing procedure i& a 
feature of some more sophisticated calculators and reflects a more sophisti- 
cated understanding of internal registers. 



Insert Table 5 about here 

* 

Non-standard sequences: Two consecutive operators . Another non-standard' 
sequence is to have two consecutive operators, such as, 

• 2 + +y= , * . 

\Z X X = 

The same strategies wer^obtained as in the previous section. One subject 
t thought the 'display would be reset, e.g., the answers would be 0 for each 
problem. Some subjects ignored the non-standard sequence; thus the display 
would say 2 for each problem. .For example, 2 + + was treated "as if it was 2 + 
and, hence, 2 would be displayed. Finally, some subjects used an incrementing 

strategy; £.g., 2 ++ = could be interpreted as 2 + 2 + 2 * thus yielding an 

i * 
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answer of 6,- and 2 x x * could be interpreted as 2 x 2 x 2 « yleldiiig an answer 

* * » 

of 8. ,A .variation of this strategy is to treat 2 + + = as 

2+ 2 » 4 and 4+4=8 yielding an answer of 8; similarly 2 x x = is treated 

as 2 x 2 = 4 and 4 x 4 « 16 yielding an answer of 16* These differences may be 

formalized in terras of Tiow the internal regxsters are evaluated and used (see 

Mayer & Bayraan, 8). Table 6 suraraarizes these strategies, and shows that most 

subjects opted for ignoring the non-standard sequence,' but experts were fair 

more likely to conceive of incrementing operations^ Since incrementing is a 

feature of more sophisticated operating systems, this difference between ex- ♦ 

' perts and novices" is sensible* 

„ ♦ * »« 

Similar results were obtained foe sequences 3uch as, 

2 x + 3 . . " 

2 + x 3 = 

o 

Most subjects ignored the first operator, yielding .answers of 5 and 6 respec- 
tively* Some subjects reset the display, often yielding an Answer of 0 for 
each problem* Some subjects used the incrementing strategy, e*g«, with answers 
of 7 and 12, respectively* there was also a subject who ignored the second 
operator, yielding answers of 6 and 5, Respectively; and there was a subject 

who preferred multiplication to addition, yielding answers of 6 to both prob- 

i • 

leras. The proportions of ignore, reset, and increment conceptions for novices 
and experts; v/ere quite similar to those sl\own in Table 6» 



Insert Table 6 about here 



Non-standard sequences: Operator after equals * Suppose the following key 



strokes wete entered, ' ' ^ \ 



• 57 

2 x =* x * * ^ . n 

J 

Subjects who use the ignore conception of nonstandard sequences act as if this 

* 

sequence is 2 x, thus the answer is 2. Subjects who use the reset strategy 

give 0' as an answer. Subjects who use an increment strategy, give answers like 

*. 

8, 16 or 4 depending on the particular/ kind of incrementing system and the 
subject's conception of when an expression is- evaluated. Table 7 symmarizes 
these three strategies and shows that while most subjects r^ly on the ignore 

o * 

conception, a substantial minority of experts rely on increment strategies and 
a % substantial minority of novices rely on a reset strategy. 
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Production systems . One goal of the study was to formally describe the 
intuitions of each subject as a list of 13 productions, i.e., 13 condition- 
action pairs. The left side of Table 8 (or 9) lists the 13 conditions that 
were present in the 88 problems we asked subjects to solve. For example, // 
after + means "pressing a number key after pressing an equals key" such as the 
last two keystrokes in the sequence 2+3. The preceding sections have summar- 
ized the different possible actions in general terms. The^ right side of Table - 
8 (or 9) gives the actions that may be associated with each condition. Actions 
are indicated as changes in the display (represented as ,D) or in the register 
(represented as R). 

Table 8 represents one of our novices. 6 The subject evaluates t only when an 

equals key is pressed (as indicated by P7), but does not evaluate an expression 

when a number key is pressed (as In P2 and P10) nor when an operation key is 

pressed (as in P4 and Pll)9 The subject evaluates an arithmetic, chain in 

« . j, • 

order from left-to-right ; thus, for each action involving eval the procedure 
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is left-to-right. The subject ignores all non-standard sequences (as indicated 
in P5, P6, P8, P12,. P13,* P14, P15, P16). For example, on the problem, 
2 + 34-7 = 

& s 

the subject begins by setting D = 2, R - 2. Then for the + key, P4 £ays that 
D = 2, and" R = 2 +. For the 3 key, P2 says that D** 3 and R = 2 + 3. For +, 
P4 says D = 3, R = 2 + 3 +• " For 7, P2 says that D = 7 and R = 2 + 3 + 7. 
Finally, whfcn » is pressed, P7 says D = 12 and R = 12 . % As another example, 
consider the problem * ^ 

7 + + = - 

First, D »■ 7 and* R = 7." Then when the first + is pressed, P4 results in D * 7 
and R = 7 +. jfken, when the second + is pressed P5 results in no change so D 
« 7 and R = 7 +• Finally, when - is pressed, P8 results in evaluation of 7 + 
which is 7: thus D = 7 and R = 7. 

Table 9 represents one of our experts. The subject evaluates when an 
operation or an equals key is pressed (as in P4 and Pll) rather than waiting 
for an equal key to be pressed (as in P7), but does *hot evaluate for pressing a 
number key (as in P2 and P10), The subject evaluates an arithmetic chain 
by performing multiplication b<*fc/re addition; thus for each action involving 
eval the procedure is multiply before add* The subject \ises an incrementing 
procedure for most non-standard sequences (as indicated in P5, P6, P8, P12, 
P13, P14, Pl5, P16). For example, on the problem 
' 2 + 3 + 7 = * 

the subject begins by setting D = 2 and R = 2. Then for the + key, P4 says 

0 

that D ■ 2 and, R = 2* +• After 3 is pressed, P2 says D = 3, R = 2 + 3, After + 

i < 

is press^J, P4|says that D = 5 and R = 5 +• For 7,^ P2 yields D = 7, 

R = 5 + 7. Finally, when^ is pressed P7 yields D = 12, R = 12 • As another 
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example consider the problem, 
7 + + - 

First, we begin with D = 7 and R - 7. For the first +, P4 results in D = 7 and 
R = 7 +• For the second +, the number in the diSplay (7) is added to the value 
in the register (7) to yield a value of 14 so D = 14 and R - 7. For the 
equals, the number in the display (14) is added to the number in the register 
(7). to yield 14 so D = 21 and R = 7. 



Insert Tables 8 9 about here 



/ ■ ( ■ 

General Summary and Recommendations * 

The results show # that even though people are ab*Le to use their calculators 
to solve arithmeticjproblems, there are important individual differences in 
people's understanding of calculator language* Ii* this paper, we have suramar- , 
ized differences in people's conception of when to evaluate an expression 
(immediately when a number key is pressed, when an operation key is pressed, or 
when an equals key is pressed), how to evaluate an arithmetic chain (left-to- 
right, multiplication-bef ore-addition, etc.), how to evaluate non-standard., 
sequences (ignore, reset, and increment). * k 

. In addition, there was a tendency for experts to differ from novices in ■ 
the following ways* (1) Experts were more consistent than novices. (2) Experts 
tended to evaluate expressions when an operator key w^as pressed more than 
novices. (3) Experts tended to evaluate ultiplicatidn-bef ore-addition in a 
chain more than novices. (4) Experts tended to increment the display for non- 
standard sequences more than novices, Thus, the present paper. provides some 
evidence that it is possible t;o describe user's conception of how calculator 
language works; in fact, in another paper [8] we provide production model 
representations for each subject. The fact that .people have different 
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conceptions of calculator operation, and that experts tend to develop more 
sophisticated ideas than novices, have implications for the design of cal- 
culator operating systems and instruction* 

In a sense, this paper has been a plea for the use of "cognitive ob- 
jectives" as well as "behavioral objectives" in users 1 learning of calculator 

languages. We need to be able to specify what we want the user to know about 

* 

how the language works, as well as what we want the^ user to be &ble to do . The 
transaction approach provides a technique^ or describing the knowledge that a 
user currently possesses, and the knowledge that we would like the user to 
acquire. One implication of this approach that warrants further study is that 
explicit training in the objects, locations, and operations (perhaps using a 
concrete mod^l of the calculator) will enhance development of oar^ognitive 
objectives. The data presented in this paper are preliminary, but they provide 

clear evidence that people's knowledge can be described (based on simple 

* <* * * 

prediction tests) and there are large individual differences among users in 

what they "know" about calculator language. 

The following recommendations are based on the idea that there should be 
as close a matcb^as possible between the user's conception of how the calcu- 
lator should >operate and the actual operating system of the calculator* 
Each recomnfendation, should be viewed as a tentative hypothesis that is 
subject to much future research, rather than a fa:ct that has been es- 
tablished through existing research. 

(1) Choose a calculator that corresponds to the intuitions of the user * 
The most obvious recommendation is to choose a calculator that works the way 
tha£ the user thinks a calculator should work, i.e". , match the characteristics 
of tjie machine to the ^ntuitiojis of the .user. In our study of 33 novice and 33 
experts, we found that Texas Instruments calculators gave answers that ;were 
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most consistent with answers given by our subjects. However, there were dis- ' . 
agreement between our subjects* answers and TI*s answers on about 20% of the 
problems for both experts and novices. Rockwell and Sharp gave even poorer 
matches to our subjects' 1 performance on the problems tfe used [see 8]. Far % 
more study is required using more problems, different types of* calculators, 
and more subjects, before any definitive conclusions can be made concerning 
which calculators have the most intuitive operating system. Thus, it is 
beyond the scope of this paper to provide endorsements for specific calculators. 
However* the present study suggests that we cannot rely on choosing an "intuitive" 

* a * • 

calculator as the solution to all our problems, because even the best fitting 
calculator (i.e., in this case, TI) is considerably different in performanc 
from what, our subjects expect. Thus, there is need for instruction that helps 
produce user conceptions that are more consistent with the way calculators 
work. 

(2) Instruct users in the concepts underlying calc ulator language* In p/ 
particular, users should be able to relate each botton push to a series 
transactions, i.e., to a description of events taking place in thWisplay 'and 
registers. The transaction approach advocated earlier [6] for BASIC appears 
to apply equally well to calculator language. The locations should be made 
explicit and visible to the learner, perhaps by providing an erasable "score- 
board" for the display and internal registers. * For each press, the learner 
should be able to alter the contents of the internal registers and display in 
accordance wifh the actual transactions. 

3. Provide diagnostic tests and remediation based on the user's 'under- 
lying concepts . The present study has shown that although two users may be 
equally proficient at using their calculators to solve standard arithmetic 
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problems,., they may differ greatly in their conception of the calculator's oper- 
ating system. Thus, a purely performance-based test of calculator skill does 
not tell a teacher what the user "knows". Instead, diagnostic tests shpuld bq 1 
carried out at *the transaction level, e.g., asking the user to state what„oper- 
ations are applied to what objects at which locations in the calculator for 
each key press. Then, remediation can be provided at the transaction level. 
For example, if a user indicates that the register ^cleared to zero for any 
non-standard sequence, this can be corrected by showing exactly what happens at 
a transaction level. Again, a concrete model (such as erasable display and 
registers scoreboard) could be used. ^ 

4. Challenge users to develop procedures for complex problems . Once 
users ,havp mastered the basics orcalculator language* and have developed 
appropriate conceptions of the underlying transactions, students, should be 
encouraged to transfer their knowledge to more challenging problems. For ex- 
ample, a student who understands the ^transactions involved ixi "incrementing 
display" could be asked to figure a way to multiply a series of numbers, kach 
by the same constant. Or, the user. could be given a problem that involves a 
geometric progression, etc. Many exercise books ar£ available [10, 14], but 
few provide any training, on the principles underlying successful performance on. 
creative problems. ( * 

5. Build on calculator* language as a means to teach other languages such 

m * 
as BASIC . Students have intuitions about how calculator^ work. This study has 

shown that the intuitions of any individual user are fairfy consistent, k 

teacher can build on these intuitions, and use them for transfer to programmable 

"calculators, to BASIC and other languages. 
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6, ' Use the calculator as a starting point for the development of computer 
- , * "*~*~ 

^literacy * ~ Through interactions with calculators the user may develop either % t 

black box or a glass box approach to computers* It is important to start early 

in helping children (and adults) see % that calculators can be understood, for. 

such an .attitude is likely to transfer to other human/machine interactions. 

Mastery of the concepts und^rl^ng calculators is just a first step dortn thV 
♦ 

road to computer literacy. , • „ . 
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♦ . Footnotes 

.This work was supported by" grant NIE-G-80-0118 from the National Institute 
of Education, Program in. Teaching and Learning. . Requests for reprints should 
be sent toi . Richard E. Mayer, Department of Psychology, University o| California 

Santa "Barbara, CA ,93106/ *• ■ " 

^ ~>< € >• - 

1. « We use the term "operating system! 1 in the most general sense to 

/» ♦ 
refer to a program that establishes the mode of user-machine interaction and 

provides for v ef ficient control pf system components. £hUs, in the present 

article, the terms "control program" or "instruction set 1 ^ could bCTsubstituted 

for "operating system". The way we "use the term in this article does not fit 

the strict definition of "operating system", i.e. a system for mediating atuong 

the demands of multiple users in a time sharing system. 

2. The main difference between experts and novices is that all of the 

1 experts had formal instruction in computer programming and had ^some introduction 

to operating systems while none of the novices did. As might be expected, 

there were other demographic differences between the groups: experts were 

older, t(60) = 2.66, p.^01, and experts scored higher in SAT-Mathematics, t 

f -4; « A. 67, p < .001. Thus; while the main comparison vas f between "liberal 

,% 

arts" Students wh6 had no formal programming experience and "engineering" 

students who had formal training in programming, any comparisons between the 

** * * 
two groups must be made in light 4 of other group differences such as age and 

SAT scores. Individual analysis of the performance of. novices who scored 

* * * • -« 

high in SAT revealed that they did not perform any more like the 'experts as 
compared to the novices who scored low in SAT. ° ° 

3. The categorization of subjects as indicated in Tables 3 through 7, 

* -/ * 

was based on an analysis of all the problems (i.e, 88 responses) rather than 

9 „ ' 
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just the few examples given in the text of this report. Subjects were classified 
using a forced choice procedure, so that each subject was placed into the 
category that was most consistent with data that he or she provided us with. 
Chi square tests were conducted pa the data i$ each of the Tables 3 through 7, 
usi^g a 2 x 2 contingency table and Yates corrective. *The expected frequencies 
were based 6fi the overall mean for each category, and tested the null hypothesis 
that there was no difference between experts and novices in the pattern of 
category frequencies. 

4. For example, the most frequently owned calculators were Texas Instruments 
Rockwell, and Sharp. The answers given by each of these calculators for each of 
the 88 problems was compared to the answers given by each subject. Difference 
scores were computed by counting the number of times that the subject gave an 
answer, that was different from a given brand. For novices who owned TI 
calculators the difference score was the lowest for TI (8.0) and the highest 

for the Rockwell (20.0), Sharp (14.1) models being in between. However, 
for students who owned calculators other than TIs the same pattern was 
obtained with the lowest difference score for TI (9; 8) and higher scores 
for Rockwell (21.6) and Sharp (15.8). A similar patteriX^s obtained 
among experts: for TI owners and non-owners their predictions most 
closely corresponded to the performance of a TI calculator rather than 

/ 

other 'brands. Analyses of variance indicated no differences between TI- 
owners and owners of other brands. 

5. There are 13^ rather than 16, productions because productions 
PI, P3, and P9 were never incorporated to the" 88 test problems. 

"6. Tables 8 and 9 describe production systems for actual individual 
subject rather than composite. 
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Table 1 

, Sixteen Elementary Calculator Commands 



Name 


Command 


Example 


Description 


PI 


// 


after // 


2 




Pressing 


a number key after pressing a number key. 


FZ 


# 


after + 


+ 


3 


Pressing 


a number key after pressing a plus key. 


T) o 

r 3 


it 

0 


after * 


s 


3 


Pressing 


r number key after pressing an equals key. 


P4 




after // 


2 


+ 


Pressing 


a plus key after pressing a number key. 


T) C 

r5 




after + 


+ 


+ * 

0 


Pressing 


a plus key after pressing a plus key. 


ro 




after = 


=s 


+ 


Pressing 


a plus key after pressing an equals key. 


r 7 


— 


after // 


3 




Pressing 


an equals key after pressing a number key. 


TJQ 

ro 


S3 


after + 


+ 


i 


Pressing 


an equals key after pressing a plus key. 


py 




after = 


S 
\ 


■ss 


Pressing 


an equals key after pressing .an equals key. 


Til f\ 

P10 


It 


oft* A IT V 


A 






a number kev after pressing a times key. 


Pll . 


X 


after # 


2 


X 


Pressing 


a times key after pressing a number key. 


P12 




after x „ 


X 




Pressing 


an equals key after pressing a times key. 


P13 


X 


after =» 


=3 


X 


Pressing 


a times key after .pressing an equals key. 


P14 


X 


after x 


X 


X 


Pressing 


a times key after pressing^ a times key. 


P15 


+ 


after x 


X 




Pressing 


a plus key after pressing a times key. 


P16 


X 


after + 


+ 


X 


Pressing 


a times key after pressing a plus key. 
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-Table 2 

Some Possible Transactions in Computer Language 

* 



Transaction 


Location 


Object 


Operation 


Description 

* 


D = D 


display 


number 


no change 


No change in the display* 

f 


D - 3 


display 
display 


number* 


find 


Find the old number in the display* 




number 


destroy 


Erase it. 




keyboard 


number 


find 


Find the number that has been entered^ in the keyboard. 




axspxay 


numoer 


create 


xul new numoer xtx axspxay* 


D » R ' 


display 


number 


find 


Find the old number in the display* 




axspxay 


^nuiuDer 


aes troy 


ijra.se xl* 




regis tei 


IIUIUU t2L 


X XllU 


"EWnil t-Vio rnimKoT" fMivv^nt""! v *f n t"lio tpM sfpr fhtit" do 

J. XI ILL Lilt? IIUIUU C J. LoUJ. J. CU UXjr Ail Lll C C5IO U C JL, UU 

no|; destroy it)* 




display 


number* 


create 


Copy the number from the register into the display. 

./• : ^ 


•D * eval (R) 


display 


number 


find 


Find the old number in the display. 




display 


number 


destroy 


Erase it. 




register 


expression 


find 


Find the expression in the register. 




register 


expression 


evaluate 


Evaluate the expressioiTxn the register (but ^do not 
N - - destroy it). 




display 


number 


create 

, M ■ 


Put the evaluated value of the register in the display. 
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Table* 2 (Continued) 
Some Possible Transactions in Computer Language 



Transaction 


Location 


Object ' 


Operation 


Description 


D - 


eval (D + R) 


display 


number 


find 


Find the number currently in the display 






register 


number 


find 

• 


Find the number currently in^lie register (but do not 

alter it). 




• 


register 


Expression 


evaluate 


Evaluate the value ox tne display plus tne register* 






display 


number 


create 


Put the new sum in the display. 


R « 


R 


register 


expression 


no change 


No change in the register. 






register • 


expression 


find 


Find the old expression in the register. 


R = 


'§ • 


♦register 


exprj^ion 


destroy 


Erase the old expression from the register. 






keyboard 


number 


find 


Find the number that has been entered in the keyboard. 






* register 


number 


create 


Put the new number from the keyboard in the register.. 


R » 




register 


T 

expression 


find 


Retain the existing expression that is in the register. 






register 


operator 


create 


Place a plus sign to the right of the ^expression in 

the register. 


R » 


"R + //" 


register 


expression 


find 


Retain the existing expression that is in the register. 






keyboard 


number 


find 


# 

Find the number that has just been entered in the 

keyboard. 






register 


number 


create 


Place the number to the right of the expression in 

in thfe- register. 



c. 



Table 2 (Continued) 
Some Possible Transactions in Computer Language 



Transaction 
R = eval (R) 

c 

? 


Location 
register 
register 
register 
register 


Obi ect 

expression 

expression 

expressidn 

number 


Operation 
f%d 

evaluate 

destroy 

create 

• 


Description 

Find the current expression or number in the register. 

■ 

Evaluate that expression or numDet. 
Erase the expression tyom tne regis* ter. 

« i „ _ _j x_t_ 4_t_ ro i nn o^F t~f)p nl a number or 
Replace it with trie evaluation uj. uhc ^iu nuuiu^*. v*. 

expression. 


R =* 


eval (D + R) 


display 


number 


find 


Find the number currently 0 in the display. 


c 




register 


number 


find * 


Find the number currently in the register. 




• 


registe^ 


expression 


evaluate 


Add them together. 






register 


number 


* 

create 


Replace it with the sum. 


R » 


eval (R*+ R) 


register 


number 


find 


Find the* number in the register. 






"register 


expression 


# 

evaluate 


Add the number to itself. 






register 


number 


destroy - 


Erase the old number from the registet. 






register 


number 


create 


Replace it with the new sum. 

*■ ■ ~ \ 


R = 


0 


register 


expression 


find 


/Find the existing number or expression in the register. 






register 


expression 


destroy y 


/ Erase that expression or number. 




* * 


register 


number 


create^ 


Replace it with zero. 

« * 2 
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* 


r 


0 79 • - . 



Conception * 
Evaluate as soon as a number key is pressed 



Table 3 

Three Major [Conceptions of When to Evaluate An Expression 

Example , 
Problem Answer 



i 



'2 + 3 
2 + 3 + 
2 + 3 - 



5 
5 
5 



Proportion of Subjects 
Novices Experts 



.21 



.06 



Evaluate as soon as an operation key is pressed- 



2 + 3 
2 + 3 + 
2 + 3 = 



3 
5 
5 



.39' 



. .73 



Evaluate as soon as an equals key is pressed. 



.2 + 3 . 
2 + 3 + 
2 + 3 = 



Note. - For 2x2 contingency table, x 2 ~ 7.44, df - 1, p < .01. 



3 
3 
5 



.39 



.21 



so 
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. ' Table 4. 

Five Conceptions of How to Evaluate an Arithmetic Chain 



Conception 
Evaluate in order from left to right. 



Example 
Problem Answer 



2+3x7=* 
2x3+7= 



35 
13 



Proportion of Subjects 
Novices * Experts 
•88 .70 



Evaluate backwards from right to left. 



2+3x7= 
•2x3*7= 



23 
20 



.03 



.00 



Evaluate only the last computation. 



Evaluate multiplication before addiiton. 



2+3x7= 
2x3+7= 

2+3x7= 
2x3+7= • 



21 
10 

23 
13 



.03 



.03 



.00 



\ 



.30 



Evaluate addition before multiplication. 



2+3x7= 

* K 

2x3+7= 



35 
20 



,03 



.00 



Note. - For 2 x 2 contingency table, x 2 ~ 6.98, df " 1, p < .01. 



Table 5 

Three Major Conceptions of How to Evaluate Equals After Operator 

Proportion of Subjects 

Conception Problem Answer Novices Experts 

Ignore the non-standard sequence. 7+= 7 "! .82 .76 



Example 1 
Problem Answer 



7+= 
7x= 



7 
7 



Reset the display. 



7x= 



0 
0 



.09 



.06 



Increment the display. 



Note. - For 2x2 contingency table, x 2 = 



7-H= 14 
7x= '49 
.52, df = 1, p = n.s. 



.09 



.18 



Table 6 

Three Major Conceptions of How to Evaluate Two Consecutive Operators 



Conception ' 
^ Ignore the nonstandard sequence* 

Reset the display. 

Increment the display. 



Example 
Problem Answer 



2-H^= 
2xx= 

2++= 
2xx= 

2++= 
2xx* 



2 
2 

0 

6'or 8_ 
8 or 16 



Proportion of Subjects 
Novices Experts 



♦ 85 



.73 



Note* - For 2x2 contingency table, =» 1.53, df » 1><,P ° n.s. 
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Table 7 

Three Major Conceptions of How to Evaluate Operation Following Equals 



Conception 
Ignore the non-standard sequence. 

Reset the display • 

Increment the display. 



Example 
Problem Answer 

2x=x 2 
2x=x 0 ' 



Proportion of Subjects ' 



2x=x 



4 or 8 or 16 



Novices 

• 85 

• 15 

• 00 



Experts 
.82 

.00 

.18 



Note. - For 2x2 contingency table, x - 4.58, d^ = 1. p < .05* 
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• Table 8 
Production System for Subject N 



Production 






• 




Number 


Condition 




Action 


P9 

JTXr 


If 


// after 


+ 


then 


Set D=//, Set R= ,f R+F 


P4 


If 


+ after 




then 


Set D=D, Set R=eval (R)+ 


?5 


If 


+ after 


+ 


then 


Set D=D, Set R-H=R+ 


P6 


If 


+ after 




then 


Set D=D, Set R=R+ 


P7 


If 


= after 


it 


than 


Set D=evgl (R), Set R=eval 


P8 


If 


= after 


t 

T 


then 


Set D=eval (R), Set R=6val 








!S 




P10 


. If 


// after 


X 


« then 


Set D-#, R = 'W 


Pll 


If 


x after 


JL 
It 


then 


Set D=D, Set R=eval (R)* 


P12 


If 


* after 


X 


then 


Set D=eval (R), Set R=eval 


P13 


If 


x after 




then 


Set D=D, Set R-R* 


P14 


' If 


x after 


X 


then 


Set D=D, Set R=R* 


P15 


If 


+ after 


X 


then 


Set D-D, Set R*=R+ 


P16 


If 


x after 


+ 


then 


set D=D, Set Rf=R* 


NOTE . — 


Subject evaluates 


for equals 


sign only; subject ignores 



Description 

Delayed evaluation and display. 

Delayed .display and immediately evaluated register. 

No change in display or register. 

No change in display, plus added to register. 
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Delayed evaluation and display. 

Delayed display and immediately evaluated register. 

Immediate evaluation and display. 

Delayed, evaluation and-dispiay-j— ~ ~ *— " ~" " 

No change in display or register. m 

Set register sign to add. * 

Set register sign to multiply. 



side of an equality means that the entire expression is held in the register; eval (R) on the right side 
of an equals means that a single value is substituted for the expression previously in the register. 

4 
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Table 9 

Production System for Subject E 



Production 



Number 


Condition 


Action 


Description 


P2 


If // 


after + 


then 


Set D=//, Set R=»"R+ff H 


Same as subject N. 


P4 


If + 


after // 


then 


Set D=eval (R), Set R=eval (R) 


Immediate incrementing display* 
* 


— ?T" 


If + 


after + 


then 


Set D=eval (D+R), Set R=R 


Immediate incrementing display. 


P6 


If + 


after « 


then 


Set D=D, Set R=R+ 


Same as subject N. 


P7 


If » 


after // 


then 


Set D=eval (R), Set R=eval (R) 


Same as subject N. 


P8 


If = 


after + 


then 


Set D=eval (D+R), Set R=eval (D+R) 


Immediate incrementing display and register. 


P10 


f 

.If 1 


nfter x 


then 


Set D=//-, Set R= u R*/if" 


Same as Subject N. 


pi l 

JTXX 


If X 


after // 


/ 

then 


Set D=eval (R), Sst R=eval (R)* 


Immediate evaluation and display. 

\ 


~~~P12 


If = 


after x 


then 


Set D=eval (D*R), Set R=eval (D*R) 


Immediate incrementing display and register. 




If X 


after - 


then 


Set D=eval (D*R) , Set R=eval (D*R) 


« 

Immediate incrementing display and register. 


P14 


If X 


after x 


then 


Set D=eval (D*R), Set R=eval (D*R)* 


Immediate incrementing display and register. 


P15 


If + 


after x 


then* 


Set D=D, Set R+=R* 

* 


Same as subject N. 

* * 


P16 


If X 


after <+ 


then 


Set D=D, Set R*=R+ 


Sam<£ as subject N. 



NOTE. — Subject evaluates for operation or equals sign; subject increments for some non-standard sequences. 
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CHAPTER 3 

DIAGNOSIS AND .REMEDIATION OF BUGS IN BASIC 



/ 

\ 



Note X \ ' 

\ 

The material in this chapter was published as the following article: 

Mayer, R. E., & Bayman,\P. Users 1 ^^conceptions of BASIC computer 

programming statements . Communicators of fche' Association for 
^ Computing Machinery , in press. * 
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Abstract 



In the process/of learning a computer language, beginning programmers may 
develop mental models for the *lapguagie* A mental model refers to the user^s 
conception of" the "invisible" infdrmation processing ttfat occurs inside the 
computer between input, and output • In this study, thirty undergraduates 
learned BASIC through a self -paced, mastery manual and simultaneously had 
hands-on access to an Apple II computer. After instruction, the students were 

tested op their mental models for the execution of each of nine BASIC v m 

L 

statements. The results show that beginning programmers — although able to * 
perform adequately on ipastery tests i» program generation — possessed a wide 
range of misconceptions concerning the statements they had learned . For 
example, the majority of the beginning programmers had either incorrect . 
conceptions for or no conceptions of statements such as INPUT A, READ k> and 
■ PRINT C. This paper presents a catalogue of beginning programmers' 
conceptions of "what goes on inside the computer 11 far each of nine BASIC 
statements. * * 

-Key Wor.ds and Phrase ; 'programming, BASIC, novices, man-machine interface. 

j 

CR Categories : 1.5, 3.66, 4.2 
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. I, Introduction 

* The main focus of this papet concerns "what is learned" when a beginning 
user is taught a computer programming language such as BASIC. The outcome of 
learnirfg can be viewed in two distinct ways: (1) Learning BASIC involves the a 
acquisition of new. information and new jrules, such as when to use quotes in a 
PRINT statement oi^how^ to produce a conditional loop using an IF statement. 
(2) Learning BASIC involves the acquisition of a inental model, such as the 
idea of memory spaces for holding* numbers. 

The present study explores the idea that learning of BASIC involves more 
thap the acquisition of specific facts, rules, and skills. Begiftning . 
programmers also develop mental models for the language in the process of 
learning the essentials of BASIC. Moran^6) suggests that the user develops a 
"conceptual moddl" of the system as he or she learns to use it. Mo ran defines 
the user t s conceptual model as the knowledge that organizes how the system 
works. Users 1 models may not be accurate or useful representations of "what 
is going on ifaside the computer. 1 ' However, most instructional effort is 
directed solely at helping the learne^ acquire the new information and 

behaviors without giving much guidance to the learner 'for the acquisition of 

'I 

useful mental models. 

Mayer (3) has suggested a framework for describing * the internal 

transformations that occur .for elementary BASIC statements. In particular, 

any BASIC statement can be conceptualized as a list oi transactions. A 

« 

transaction is a simple proposition asserting some action performed, on some 
object at some location in the computer. For example, LET D = 0 involves "the. 
following transactions: Find the number is memory space A. Erase the number 
in memory space A. Find the number indicated on the right of the equals^ £ign. 
Write this number in memory space A. Find the next statement in the program. 



Experts and novices are likely to differ x*ith respect to their mental 
models for programming statements. For example, an expert programmer may have 
developed an accurate conception for a counter set LET, such as the one given 
above*. However, " the novice qiay lack a coherent mental model or may possess 
incorrect conceptions for BASIC statements. In a rec&nt study, Mayer & Bayman 
(5) found that novice and expert calculator users differed greatly in their 
conceptions of "ttfhat goes on inside the calculator" for various key presses. 

How "much training one needs to acquire a conceptual model (or mental 
model) has not yet. been explored in research. A first step in, addressing this 
issue is to studir novice users 1 understanding of newly learned programming 
statemencs. In this paper, we describe novice users 1 conception^ of nine BASIC 

statements, using a transactional analysis, m + 

• *» 

II. Method 

The subjects were 30 college undergraduates who had. no prior experience . 
with computers or computer programming ^ The subjects took a modified version 
of a self-rinstruction, self-paced, mastery course called BASIC' in Six Hours - 
'(2) that is widely used for teaching BASIC iVthe Microcomputer Laboratory of 
the University of California, Santa Barbara. The instruction occurred over 
the course of three sessions and involved both a programmed manual and 
hands-on access to an Apple-II computer. Subjects were required to pass^ 
mastery tests over one section before moving on to^the next section of the 
manual. 

, Following successful completion of the course, the users were given a 
procedure specification test. For each of ni^e statements (from the 

instructional course) subjects were asked to write, in plain English, the 

o 

steps that the computer would carry out for each statement, Us ^ s were 
instructed to write each step on a separate line of the test sheet. The 
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method also, involved additional tes,ts which are described in a more detailed 
report of this sfcudy^fl)./ ^ 
III. Results ' - 

A. , Scoring ^ t . 

i . Each user's protocol for each" of. the nine statements was broken down into 

a list o£ transactions by two scorers, as described in an earlier paper (3). 

Three types of transactions were 'observed for each statement:' (1) cor- 
rect transactions — For example, the key correct transaction for LBSC A = B + 1 
is store the value of B 'plus 1 in memory space A", (2) . incomplete . 
transactions — Foy example, a user's answer foy LET A = B + 1 could include 

"store tha value of B + 1 in memory." (3) incorrect transaction — For example, o 

> 

a user f s answer for LET A = B + V could include "store the equation A = B + 1 
in memory." . ^ 

B. Differences among statements . 

In order to make comparisons of users 1 conceptions among the nine 

statements, each user f s verbal protocol for each statement was categorized as 
r t 

correct, incomplete, empty, or incorrect. \^The criteria for classifying 
c protocols were: correct— rif the user f s protocol included the key correct . 
transaction (s) and no incorrect transactions, incomplete — if the user produced 
incomplete versions of the key transaction(s) and no incorrect transaction, 
and empty — if the subject produced no correct or incomplete version of the key 
transaction (s) and no incorrect transaction, and incorrect — if the subject \ 



produced one or more incorrect transactions. Table I presents a summary of 
the percencage of users who produced each type of conception for each of the 
nine BASIC statements. Table I presents the nine statements in order of 
'difficulty based on proportion cqrrect conceptions. 
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C. Frequency of Misconceptions / - ' 

■ f * 

For each of the nine statement^ a frequency tablfe was generated by ^ 
tallying the number of occurrences of each correct, incomplete, and incorrect 
transaction in the protocols pi the 30 users. Table II lists the key correct^ 
transaction (s) and the threje most common incorrect or incomplete transactions 

. / - - • • - 

£i»e., misconceptions) for each of the ijine statements. 



Insert Table II about here ^ 



»IV. Summary and Recommendations * • 

Users 1 lack of understanding and misconceptions c^n be summarized as 
follows: 

1. INPUT Statements . Users, have difficulty in conceiving where the 

■ * . 4 

to-be-input data comes from (i.e., the keyboard) and how it *is stored in 
memory (i.e., in the indicated memory space). Furthermore, many users fail to 
understand /the nature of executive ^controlr-i.e. , that the computer will 
"wait", for input from the keyboard as cued by the question mark on the screen. 
A major misconception is that "INPUT A" means that the letter A is input and 
stored in memory. These users need explicit training qoncerning the role of 
the input terminal, the wait-run control, and the memory* spaces. « 

, 2. READ-DATA Statements . Users 'have difficulty in conceiving of where 
the to-be-read data comes from (i.e., the input queue or DATA statement) and 
how it is stored in ^memory (i.je., in a specified memory space). For example, 
a major misconception is that "20 DATA 80, 90, 99 u means that the numbers are 
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placed into memory or printed on the°screen; another major misconception is 

that "30 tfEAD A" means to find the value of A and print that value on the 

screen. Subjects need explicit training concerning the data stack and memory. 

spaces* * ' 

■ ♦ * * 

3. Conditional and Simple GOTO Statements . Users 1 major difficulty with 

t 

the GOTO statement is that they do not understand what will happen next, afftei; 
• program execution m^ves on to the desired line^ Also, with the conditional 
GOTO, users have difficulty in understanding what to do .if the conditioners 
false. Misconceptions include thinkin/; that "60 GOTO 30" means to find the 
number 30 (rather than line 30) and fl IF A < B GOTO 99" means move to line 99 
(Without a, test). Hence, beginners need training .in executive control of the 
order of execution^o^f statements in a' program, 

4* LET Statements T JJsers seem to get confused between solving or 
storing an equation (i.e., treating ^the equal sign as an equality) and making 
an assignment. *Those who. seem to understand the assignment property in the 

4 ^ 

statement still have difficulties* in conceiving where to store the assigned 
value. For example, 3*major misconception for "LET A = B & 1" or "LET D « 0" 
is to thrnk that the equation is stored in memory. Beginning programmers need 
explicit training concerning memory locations and under what conditions values 
stored in those locations get replaced. 

5. PRINT Statements . Users seem to confuse the function of PRINT C and 
PRINT ^"C". Also, users have difficulty in^ conceiving that these statements 
simply display on the screen whkt is asked to be printed; users incorrectly 
j assume that the computer keeps a record of what is printed somewhere in 

- me&.ory. Beginning programmers need explicit training in comparing the types 

* of PRINT statements. • 
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The present study provides evidence that "hands-on experience 11 is not 

sufficient for the productive learning .ot computer programming by novices. 

Users tend to develop conceptions of the statements that either fail to 

• - ♦ 

include the main idea or that include outright misconceptions. Explicit 

training is needed including the introduction of a concrete model showing the 

key locations in the computer (e.g., memory spaces, input stack, etc.), verbal 

and visual descriptions of the key transactions for each statement, and 

encouragement of the user to role play "what the computer is doing" for 



statements and programs. These techniques are reviewed elsewhere' (4). 
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Table I 



Percentage of Users with Correct, Incomplete, Incorrect, and Empty 

- 4 $ ' 

Conceptions for the Nine BASIC Statements i 



Conceptions 



Statements 


Correct 


Incomplete / 


Incorrect j 


Empty 


INPUT A 


„ 3% 


30% 


30% 


37% 


30 READ A 


10%' 


27% 


17% 


47% 


IF A < B GOTO 99 


27% 


. 27% 


40% 


7% 


LET A = B + 1 


. 27% 


10% 


60% •• -4 


3% 


20 DATA 80, 90, 99 


27% 


17% 


13% 


43% 


60 GOTO 30 


. 27J„ 


56% ' 


10% 


, 7% 


PRINT C 


33% 


0% 


47% 


20% 


LET D = 0 


43% 


3% *x, 


53% 


' 0% 


PRINT "C" 


80% , 


V ^ 


13% 


7% 
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•Table II .» 

V 

Percentage of Users Who Produced Key Correct and Incorrect or Incomplete 
a Transactions for Nine BASIC Statements 



'30 READ A 
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Percentage of 



Transaction ^ • Users 



INPUT J* 

*Print ? on the screen, 7% 
' * Wait for a number and <CR> to be entered 23% 
*Write "the entered number in memory space A. 3% 

\ 



Write A in memory (or a data list), 30% 



Wait for a>certain number or letter, *" - 13% 

Print A on 'the screen. • 3% 



, *Write next number from DATA in memory space A* 10% 

Print value of A on the screen. ^ 10% 

Writja letter A in memory, \ -3% 

U . • • - - 

Wait for number to be entered from keyboard 3% 



IF A <-B- GOTO 99 

* • • 

*If the value of A is iess than B, then move 63% 
to line 99 in the program.* 

*If the value of A is more than or equal to B, then 33% 
♦ move on to the next statement in the program. 

Move to line 9*) (without tejt) • 10% 

Print the number 99 or line 99 on the screen. 10% 

Write A or B or A<B or a number inj memory ." J.0% 



TABLE II (continued) 



v Transaction • « , 

. . . • , 

LET A = B -U 

*Write the obtained value in memory space A. 
Write the equation in memory. - 

k 

Write B + 1 in memory space A. 

Print A=B+1 or A or the value of A on, the screen, 
20 DATA 80, 90, 99 

-jfeput numbers in input/queue. 
Put numbers in memory., 

Print numbers on screen. * 
Put numbers in memory space A.. 



Percentage gf 
Users 



30% 
43% 
33% 



27% 
20% 
13% 



60 GOTO 30 

*Move to line 30 in the program. 
^Continue from there. 
Find the number^ 30. " 

Move to line 30 if A- does not 'equal a. 'certain value. 
Print line 30 on screen. 



67% 

37% ' 

10% 

\ 
7%. 

3%' 



PRINT C 



*Print number (or *0) 6n screen. 

<? » 

Print the letter C on the^cfeen. • 
- Write the letter* C in memory. 
Print either "error" or nothing on the 'screen. 



40% 
33'% 
7% 
7% 



TABLE TI (continued) 



Transaction 



LET D = G ' 



*Jtfrite zero in memory space, p. 
Write the equation in memory,. 
Write, D or 0 in metnory. 
Print the equation on the screfeu. 



PRINT "C" 



*P,rint the letter C on the screen/ 

» ' S 

Write the letter C in' memory. 
Print vt he value of*C on the'sdreen. 
Find the number in memory space Op 
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Percentages of 
Users 



■ kit 
' 47% 

; 13% 

77. 



83% 
7% 
7% 

.3% 



fiote. — Astfcriskp (*) indicates ktey correct transaction for each statement . 

Thejthree most.commpri incorrect or incomplete transactions are. also 

* * , " s \ 

listed for each- statement. * I 
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